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Carol L. Macnee 
Roy Penchansky 


Targeting Ambulatory 
Care Cases for Risk 
Management and 
Quality Management 


Means are needed to identify ‘‘highly productive’ areas for quality review and risk 
management in ambulatory care; untargeted medical record reviews have too low a 
yield and too high a cost to be useful. Highly productive areas are those with 
important medical and economic consequences, large numbers of cases, and a 
reasonable potential that problems exist. This paper describes untoward event screens 
to identify highly productive areas for review based on hospital discharge diagnosis, 
procedure codes, and parameters such as length of stay and cancer staging. The 
approach proposed has been evaluated in six ambulatory care organizations and 


appears effective and efficient. 


To evaluate and influence quality of care, and to 
identify cases where a poor outcome may require 
action to ameliorate negative clinical, legal, or fi- 
nancial consequences, health care delivery systems 
must identify bad outcomes—untoward events, 
under- and overutilization, and poor proficiency. 
Given the time and costs, and the low yield of cases 
with quality concerns that derive from clinical/med- 
ical record reviews (Thomas and Longo 1990), there 
is a need to identify ‘‘highly productive’’ areas for 
review, as well as to develop approaches to case 
identification in these highly productive areas (Han- 
nan et al. 1989). Highly productive areas are those 
having important patient consequences, such as the 
‘‘sentinel events’’ originally described by Rutstein 
et al. (1976), large numbers of cases, and a reason- 
able potential that problems exist (Benson and 
Townes 1990). 


Approaches to Targeting Cases 


A number of approaches to targeting highly produc- 
tive cases for hospital quality assessment have been 


or currently are being examined. In particular, there 
are occurrence screening approaches that employ 
diagnosis related groups (DRGs) and The Inter- 
national Classification of Diseases, 9th revision, 
(ICD-9) codes, along with selected case character- 
istics. These approaches include the Health Care 
Financing Administration’s (HCFA’s) sampling cri- 
teria (HCFA 1988), disease staging (Aronow 1988), 
and severity and acuity indexes (Thomas and Longo 
1990). Such systems allow hospital quality monitor- 
ing units to identify cases that differ from normative 
expectations in terms of length of stay, use of pro- 
cedures, and outcomes. The applications of these 
approaches have focused on areas primarily rele- 
vant to the quality of care within the hospital set- 
ting, the exception being the potential for using 
disease staging to evaluate admission appropriate- 
ness (Gonnella, Louis, and McCord 1976; Aronow 
1988). 

Are hospital ‘‘occurrence’’ reports useful to am- 
bulatory care organizations? Certainly an ambula- 
tory care program should obtain, and would find 
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useful, any data collected by hospitals that involve 
the ambulatory care organization’s physician or 
patient. These provide, however, a narrow charac- 
terization of the types of cases meriting review, 
whether for risk management or quality improve- 
ment. Most hospital criteria relate to success of 
procedures or cases treated, and do not address the 
reason for the procedure, or the possibility that the 
level of difficulty of the case might represent inad- 
equate prior care, such as a delay in diagnosis, a 
missed diagnosis, or a lack of preventive or well 
care. 

As examples, a hospital occurrence screen might 
identify an infection or death after breast surgery, 
but as long as the surgery went well it usually would 
not identify a case that involved a delay in diagnosis 
until the cancer had advanced to stage II or III. An 
admission for resetting a nonaligned or misaligned 
fracture might be identified for hospital quality as- 
surance and risk management purposes if there 
were a death, infection, drug reaction, or loss of 
function due to the treatment, but the case would 
not otherwise be targeted for review. For an ambu- 
latory care organization, a nonaligned or misaligned 
fracture alone is a prime concern for risk manage- 
ment and quality assurance activities. 

In contrast to approaches targeting highly pro- 
ductive cases in hospital quality assurance, meth- 
ods specifically developed to evaluate ambulatory 
care often have been limited to detailed review of 
specific tasks within ambulatory care (e.g., routine 
well-child height and weight checks, or blood glu- 
cose monitoring). Earlier efforts in ambulatory care 
entailed significant time and resources for imple- 
mentation (Palmer et al. 1984; Palmer 1988). More 
recent programs utilize claims data but still tend to 
focus primarily on utilization concerns (Leather- 
man et al. 1991). While these programs provide an 
important contribution to assessment of quality in 
ambulatory care, particularly in regard to preven- 
tive or chronic disease care, they do not serve to 
target a broad range of areas for quality assurance 
and risk management, and they can be costly to 
implement. An exception to the detailed record 
review or utilization review is the approach taken in 
the Minnesota Project (Solberg et al. 1990). More 

will be said about this project later. 

- Ambulatory care systems face a far more difficult 
problem than hospitals in identifying highly produc- 
tive areas for quality assurance and risk manage- 
ment, and in finding approaches to targeting cases 
for evaluation (Gonnella and Louis 1988). This is 


Targeting Ambulatory Care Cases 


because: 1) in ambulatory care there are far more 
encounters, and it is more difficult to define distinct 
episodes than in hospital care; 2) labeling of symp- 
toms or diagnoses varies more and is less precise for 
ambulatory care patients than for hospitalized pa- 
tients; 3) negative outcomes from ambulatory care 
generally do not take place while the patient is in the 
institution and may not be recorded; and 4) most 
ambulatory care data systems are not highly devel- 
oped. 

Nearly all hospitals now have computerized files 
of DRGs and/or ICD codes—with multiple diag- 
noses—and attention is paid to the quality of diag- 
nosis coding and entry. With these a hospital can 
readily use a series of screens to scan hundreds of 
discharges and identify small groups of cases that 
merit individual review. If an ambulatory care or- 
ganization employed a computerized coding system 
for complaints, diagnoses, and outcomes, then a 
targeting system parallel to that used by hospitals 
could be employed to identify key untoward out- 
comes of ambulatory care, such as missed diag- 
noses, delayed diagnoses, drug reactions or inter- 
actions, ineffective procedures, or underuse of 
defined services for at-risk patients. However, cod- 
ing systems of symptoms/complaints, diagnoses, or 
outcomes seldom are employed, and few ambula- 
tory care organizations have computerized files of 
data about ambulatory encounters. 


Development of Untoward Event 
Identification Screens 


An approach was developed to identify homoge- 
neous groups of untoward outcomes of medical care 
that were ambulatory care/primary care oriented. 
This was done as part of a research effort to examine 
the doctor-patient and system-patient relationships 
as predictors of malpractice cases when a negative 
outcome has occurred in ambulatory care organi- 
zations. A method to identify cases based on the 
usual data systems of ambulatory care organiza- 
tions could not be found for the reasons noted 
earlier, and the cost of medical record reviews to 
identify cases was prohibitive. However, since 
most large group practices or health maintenance 
organizations (HMOs) have computerized files of 
hospitalized cases (with diagnosis and procedure 
codes, as well as length of stay) as part of their 
payment or billing and utilization review systems, it 
was decided to try and employ these systems to 
identify the types of ambulatory occurrences/events 
desired. 
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The goal was to identify codes for diagnoses and 


procedures, and length of stay information, which 


would define homogeneous groups of important un- 
toward events (UEs)—events that were important 
because of their potential consequences to the pa- 
tient or their relationship to the ambulatory care 
organization’s quality assurance and risk manage- 
ment goals. To qualify as a UE required meeting the 
following criteria: 1) the UE would have had to 
manifest itself in a hospital stay (an ambulatory 
surgery encounter also could be employed); 2) the 
UE could be defined using ICD-9 diagnosis and 
Current Procedural Terminology (CPT) codes indi- 
vidually, or in combination, and length of stay; and 
3) the UE often would signal (though not necessarily 
mean) inadequate care. 

This approach is similar to severity-of-illness 
methods which have focused on quality of care in 
hospitals (lezzoni and Daley 1992; Thomas and 
Longo 1990; Aronow 1988), but it specifically tar- 
geted ambulatory care concerns. It also is similar to 
the staging methodology developed by Gonnella 
and Louis (1988), but allows direct identification of 
‘‘high yield’’ cases without the intermediate step of 
using criteria to classify cases into specified stages. 
In addition to the three criteria mentioned, research 
purposes required that the UEs had to identify cases 
with similar consequences to the patient, and had to 
have a reasonable incidence. However, for opera- 
tional purposes in an ambulatory care organization, 
homogeneity in consequences or the need for a large 
incidence would not be essential. 

Medical directors of ambulatory care organiza- 
tions, medical school specialists, and a few selected 
ambulatory care organization physicians identified 
‘‘problem’’ areas for both risk management and 
quality assurance in ambulatory care. Malpractice 
literature was another source used to identify po- 
tential problem areas that have a primary care or 
ambulatory care component. A number of problems 
that might meet the criteria for a VE were suggested 
by the physicians and literature; some already have 
been noted—nonalignment/misalignment of a frac- 
ture and breast cancer first identified at stage II or 
beyond. Others of particular interest to ambulatory 
care organizations— because they might reflect de- 
lay in diagnosis or problems with treatment—in- 
cluded: complications of ectopic pregnancy and 
abortion; appendicitis with peritonitis; rectal can- 
cer; and cervical cancer stage IB or greater. Two 
other UEs that may derive from either problems 
with prenatal care or from activities in the hospital 
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were identified because of their frequency and con- 
sequences to the patient/family or their risk to the 
ambulatory care organization. These were baby 
birth injury and death, and postpartum complica- 
tions. Several of the UEs listed earlier might be 
identified by a hospital reporting system. However, 
the focus of review in the hospital would not nec- 


-essarily include the appropriateness of utilization 


and proficiency in the delivery of ambulatory care. 

Six UEs were selected for the purposes of this 
study: appendicitis with peritonitis; rectal cancer; 
breast cancer stage II or greater; postpartum com- 
plications with greater than a three-day length of 
stay; birth injuries; and complications of abortion 
and ectopic pregnancy. The UEs included cases 
that would be handled by different types of physi- 
cians—pediatricians, internists, family physicians, 
and obstetricians. Further, these UE cases impact 
adults and children, males and females, and relate to 
well or preventive care, diagnosis, and treatment 
activities. 

For each of the UE problem areas, medical con- 
sultants defined the ICD-9 diagnosis and CPT pro- 
cedure codes, and sometimes length of stay, that 
would capture the cases desired. For the breast 
cancer cases, it was found that information on stage 
of cancer was needed to limit the cases identified to 
those whose disease was advanced enough to sug- 
gest a possibility of delay in diagnosis or in seeking 
care. Although use of cancer registry information 
involved using a type of staging, this is the existing 
staging method for cancer which is part of the usual 
diagnostic process. No criteria needed to be devel- 
oped or applied. Thus, each UE problem was de- 
fined as those cases at the intersection of specified 
ICD-9 diagnosis codes and specified CPT procedure 
codes, as well as meeting selected cancer staging 
and length of stay minimums. The staging and 
length of stay criteria contribute measures of sever- 
ity. An example of the parameters/criteria used in 
the research project for case selection is presented 
in Table 1.' 


Methods 


Six large ambulatory care organizations, all group 
or staff model HMOs with two being parts of large 
medical centers, were asked to develop computer 
programs applying the UE criteria to their hospital- 
ization files from 1986 through 1990. A number of 
sites did not have adequate information systems or 
data to employ the UE identification screens prior 
to 1988. However, all sites now have the manage- 


This content downloaded from 193.142.30.116 on Sat, 28 Jun 2014 13:09:41 PM 
All use subject to JSTOR Terms and Conditions 


Table 1. Appendicitis with peritonitis 
untoward event 


Definition: Appendicitis with either generalized 
concurrent peritonitis or peritoneal abscess. 


Identify all cases with either of the following hospital 
discharge diagnosis ICD-9 codes: 


1.540.0 Acute appendicitis with generalized 
peritonitis 


2. 540.1 Acute appendicitis with peritoneal abscess 


Which also have one of the following [CD-9 procedure 
codes: 


A. 47.0 Appendectomy 
B. 47.2. Drainage of appendiceal abscess 
C. 47.9 Other operation on appendix 


Include all cases with: 


1 and (A or B or C), or 
2 and (A or B or C) 


ment information systems and data to identify 
cases. Management information systems personnel 
reported that development of programs to retrieve 
UE cases required an initial time investment rang- 
ing from a few hours to two days. However, once 
the programs were developed they were modified 
easily to examine different sets of criteria. 

Access to tumor staging information proved to be 
more difficult. None of the study sites had cancer 
staging included in their data fields. The two sites 
that are part of large medical centers were able to 
access staging information from their hospital can- 
cer registries. Two other sites were able to access 
cancer staging from several regional cancer regis- 
tries. However, in the last two sites accessing stag- 
ing data was a great administrative burden and, 
therefore, not pursued from registries. This led to 
the need to review for staging all cases identified 
through codes as breast cancer with mastectomy 
from these two sites. Although some sites did not 
have ready access to tumor staging for past cases, it 
would not be difficult for ambulatory care organi- 
zations to acquire and store this information for 
future use. 

The predictive value of the UE screens or validity 
of the cases identified—that the cases did represent 
the type of UE case intended—was checked when, 
as part of the larger research project, registered 
nurses or medical record room staff abstracted the 
ambulatory care records of cases identified as meet- 
ing the UE screening criteria. These personnel were 
employees of the six study sites and were trained as 
abstracters in a two-day training session. Cases 
were classified as ‘‘not a UE”’ for a variety of 
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reasons, most of which related to the criteria for the 
research study and not whether the patient had 
experienced the UE. Reasons for exclusion as a UE 
case included: presence of a previous severe med- 
ical condition (such as AIDS) which made outcomes 
differ from those of most cases; care provided out of 
the geographic area specified for the study; or lack 
of ambulatory care at the study site prior to hospital 
referral of the patient with the UE condition. 

To assess utility of the screening criteria in iden- 
tifying cases that warranted quality improvement 
and risk management attention, the medical records 
for two types of UE cases—appendicitis with peri- 
tonitis and birth injury—were reviewed for possi- 
bility of management causation of the UE and for 
possibility of negligence associated with the UE. 
Since management causation is a prerequisite for 
negligence, reviewers first were asked to evaluate 
cases for evidence that the injury (the UE) was 
related, in part, to the medical care the patient 
received or should have received. If evidence of 
management causation was found, the case also was 
evaluated for evidence of negligence—that the care 
given deviated from standard practice for a similar 
physician in a similar circumstance. 

To accomplish the reviews for medical manage- 
ment causation and negligence, records were ac- 
quired for each appendicitis and birth injury case 
from the date that the first UE symptom was doc- 
umented through the dates of hospitalization for the 
UE. The blind records for each appendicitis and 
birth injury case were reviewed independently by 
two board certified physicians—internists, family 
practitioners, and pediatricians for the appendicitis 
cases, and obstetricians for the birth injury cases. 
Physician reviewers were all in active practice in 
ambulatory care, and represented a variety of prac- 
tice settings, from fee-for-service to closed panel 
HMOs to large academic medical centers in 11 
different states. 

Physicians who reviewed the cases completed a 
detailed Merit Assessment Form. The form asked 
them to rate the possibility of management causa- 
tion of the UE (described as ‘‘medical management 
or lack of medical management caused the UE or 
caused an increase in the severity of the UE’’), and 
the possibility that the UE resulted from negligence 
(defined as ‘‘deviation from standard practice for a 
similar physician in a similar circumstance’’). The 
merit assessment form (see Table 2) was based on 
the Harvard Medical Practice Study Adverse Event 
Analysis Form (Brennan, Localio, and Laird 1989), 
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Table 2. Contents of the Merit Assessment 
Form 


cases by board certified physicians. The results that 
follow include the number of UEs identified for 
review, the validity of the UEs identified, the num- 


I. Classification of the extent of injury 7 
= 3 ber of physicians whose cases were reviewed, and 
II. Determination of the cause of the untoward Bae te 
fener the utility of the UEs for quality improvement and 
III. Categorization of the strength of evidence that risk management. 


there was medical management causation 
IV. Initial evaluation of a possibility of negligence 
V. Classification of type(s) of error(s) in the case 


Vi. Determination of mitigating or aggravating 
circumstances in determining the possibility 
of negligence 


Results 


Table 3 presents the number of cases identified for 
the six UEs, for periods of one to five years of 
exposure for six ambulatory care organizations. 


VII Determination of presence of negligence The screens for the six UEs produce a small number 
VIII. Categorization of seriousness of negligence of cases for review per year, with an average of 
IX. Categorization of the strength of evidence for one-half case per 1,000 enrollee years for all sites 


negligence 


Note: Adapted from the Harvard Medical Practice Study 
Adverse Event Analysis. 


and was revised for this study. Physician reviewers 
were oriented and trained in use of the form with a 
selected practice case. Physicians used their own 
judgment regarding both medical management cau- 
sation and negligence; no protocol was given. 
Therefore, judgments were based on implicit rather 
than explicit criteria. All UE cases received two 
independent physician reviews. If the two review- 
ers disagreed regarding the possibility that the UE 
resulted from negligence, the case was submitted 
for a third review. 

In summary, both the validity and utility of the 
cases identified by the UE screens were assessed. 
Validity was checked as part of the process of 
medical record abstracting of cases, and utility was 
examined through independent reviews of blind 


and years. Five sites fell between .4 and .9 cases per 
1,000 enrollee years. Site 3 (in Table 3), one of the 
two sites with .9 cases per 1,000 enrollee years, had 
some fee-for-service UE cases that were not ac- 
counted for in the denominator, therefore inflating 
somewhat the estimate of cases per 1,000 enrollee 
years for that site. 

Analysis of 1,111 medical records abstracted for 
the research study indicate that 199 (18%) of the 
cases identified failed to meet some other criteria for 
the study, or did not actually have the UE as defined 
(see Table 4).* Reliability of abstracters’ judgments 
as to whether cases were true UEs was found to be 
high, with 98% agreement on categorization be- 
tween abstracters and the research investigator 
(Macnee), who is a nurse practitioner. 

Eighty-eight (44%) of the 199 cases that were not 
UEs were breast cancer cases, and the bulk of these 
(n = 76) were from the two sites which could not 
access tumor staging before abstracting the case. 


Table 3. Number of UEs identified for 1986 through 1990 for six ambulatory care 


organizations . 
Site 1 Site 2 Site 3 Site 4 Site 5 Site 6 
(180,000) (75,000) (60,000) (150,000) (84,000) (150,000) 
1988-1989 1986-1990 1986-1989 1986-1990 1988 1988-1990 
Exposure years 2 4 5 1 3 
Untoward event (UE) 
Appendicitis 15 36 47 80 1] 38 
Rectal cancer 12 12 32 32 0 12 
Breast cancer 25 50 81 111 5 135 
Postpartum complications 0 10 0 17 9 20 
Birth injury 26 4] 57 73 50 40 
Abortion/ectopic complications 12 qd 0 15 0 0 
Total UEs identified 90 156 27, 328 75 245 
UEs per 1,000 enrollee years Le 42 .90 44 .90 54 


Note: Enrollments are in parentheses under site numbers. 
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Table 4. Rate of error in identification of 
untoward events by UE type for combined 
cases across Sites 


Cases Nota ‘True’? Error 
identified UE UE rate (%) 

Appendicitis 227 26 201 11 
Rectal cancer 100 8 92 8 
Breast cancer 

Total 407 88 319 22 

Prestaged 222 12 210 5 
Postpartum 

complications 56 11 45 20 
Birth injury 287 66 221 23 
Abortion\ectopic 

pregnancy 34 0 34 0 
Total 1,111 199 912 

with breast 

cancer prestaged 926 123 803 


These 76 cases were excluded because it was found 
that they were not stage II or greater, though they 
were breast cancer cases. Cancer staging data was 
obtained easily if organizations knew they were 
going to employ it; therefore, we estimate that the 
error rate for breast cancer would be about 5% for 
ongoing program use. 

Another 66 cases found not to be UEs were birth 
injury cases. A third of these cases were from one 
study site (with an error rate of more than 80%). At 
the five other sites, the error rate for the birth injury 
UE averaged 13%. Two factors were probably at 
work here: first, the complexity of the screening 
criteria for this UE and, second, at the one study 
site with the high error rate there was obvious 
variability in one hospital’s use of ICD-9 codes for 
obstetrical care. 

The remaining 45 cases not included as UEs were 
distributed among the other four types of UE prob- 
lem areas and reflected nonsystematic factors which 
primarily related to the research study criteria, such 
as cases where an earlier cancer had been diagnosed 
prior to the rectal cancer. In a few cases the UE 
problem was suspected, but eventually another di- 
agnosis was established. In most instances, UE 
cases were excluded because they failed to meet 
study criteria established to promote homogeneity 
in outcomes, rather than due to an error in coding 
(the exception being the birth injury problem at the 
one site). An estimated overall error rate for the UE 
screens was less than 10%. 

Because many organizations have an interest in 
profiling physicians for quality improvement, the 
number of different physicians who were identified 
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as primary providers for the UE cases was exam- 
ined. The primary physician provider to UE case 
ratio was 51 providers per 100 cases, with a range of 
28 to 69 providers per 100 UE cases for the six sites. 
Approximately 35% (n = 129) of the physicians 
identified were found to be primary providers for 
more than one UE case. Thus, cases identified by 
the UE screens were not those of just a few physi- 
cians; however, neither were they such isolated 
instances that each represented the care of a differ- 
ent physician. 

A total of 203 cases—123 appendicitis cases and 
80 birth injury cases—have received full physician 
review for management causation and negligence. 
Data have been collected chronologically, and the 
UE cases that have received physician reviews are 
those from the earlier years covered in the study. 
All cases that had been abstracted, and for which 
medical records had been obtained by November 
1992 were included. | 

There was agreement on the presence or absence 
of medical management causation by two physi- 
cians in 169 (83%) of the 203 cases, and physicians 
agreed on presence or absence of negligence (with- 
out a third review) in 182 (90%) of the 203 cases. 
This represents a high level of inter-rater agree- 
ment, given that judgments were based on implicit 
criteria. Because the focus here is on evaluating the 
UE screens for use in quality assurance and risk 
management, any case in which at least one physi- 
cian reviewer found evidence of medical manage- 
ment causation is considered relevant and warrant- 
ing review. Of the 123 cases identified by the 
appendicitis screening criteria, 43 (35%) were 
judged by at least one of the reviewing physicians as 
having at least ‘‘slight to modest evidence’’ for 
being caused by medical management (See Table 
5).2 Of the 80 birth injury cases, 20 (25%) were 
judged by at least one physician as having at least 
slight to modest evidence of medical management 
causation. 

Twelve appendicitis cases (10%) and nine birth 
injury cases (11%) required a third physician review 
because of disagreement about the possibility of 
negligence in the first two reviews. Fifteen of the 
appendicitis cases (12% of the total cases re- 
viewed), were rated by two physicians indepen- 
dently as having at least ‘“‘slight to modest evi- 
dence’’ that the UE resulted from negligence (see 
Table 6). A total of 55 errors considered deviations 
from standard medical care were identified in the 15 
cases (there were at least two reviewers per case 
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Table 5. Ratings on medical management 
causation for appendicitis and birth injury 
cases 


Appendicitis _ Birth injury 
(n = 123) (n = 80) 
Number % Number % 


Cases reviewed as no 
causation by two 
reviewers 80 65 60 ie) 


Cases reviewed as some 
causation by one 
reviewer 20 16 14 17 


Cases reviewed as some 
causation by two 
reviewers 23 19 6 8 


and each could identify more than one type of 
error). Errors were grouped into assessment, pro- 
cess of care, performance of a procedure, drug 
treatment, system errors, or other errors. Not sur- 
prisingly, the most common type of error identified 
in appendicitis cases was an error in assessment— 
‘‘failure to order or perform appropriate screening 
or diagnostic procedures or tests’’ (46% of errors). 
The second most common type of error identified 
was in the process of caring for the UE—“‘failure to 
employ accepted standards of management”’ (33%). 

In contrast to the appendicitis cases, only two 
birth injury cases (3% of UE cases reviewed) were 
rated by two physicians as having at least slight to 
modest evidence that the UE resulted from negli- 
gence (See Table 7). A total of 10 deviations from 


care system, such as equipment, communication 
problems, or staffing.’’ 


Discussion 


The screening criteria that were developed, using 
codes for diagnoses and procedures, cancer staging, 
and length of stay information, appear to identify 
accurately cases which have important implications 
for quality assurance and risk management. Given 
the nature of this study, it is not possible to assess 
‘false negative’’ errors where ‘‘true’’ UE cases 
were not identified by the screens. ‘‘True’’ cases 
could be missed for two reasons: poor coding of 
cases in hospitals, and inadequate information sys- 
tems at the study sites. If anything, the kinds of 
errors found suggest that problems with reliability 
in use of ICD-9 and CPT codes probably reflect 
errors of commission in the direction of ‘‘overcod- 
ing’’ rather than failing to code cases, and many of 
these errors were from one site. This would lead to 
a problem of ‘‘false positives’’ rather than ‘‘false 
negatives.’’ As indicated earlier, false positives 
were excluded based on information from abstract 


Table 7. Birth injury case ratings on medical 
management causation and negligence 


Site Site Site Site Site 


1 2 3 4 5 Totals 


Number of cases 
reviewed 2 2 32020 337 7 80 


Number of cases 
with possible 


standard medical care was identified in these two medical 
ry ae management 
ae Three wai errors of assessment— al See liGn \ 0 5 13) 1 (20 (25%) 
to oy er or perform oe OpEa screening or 1ag- ber oh cases 
nostic procedures or tests.’’ Three errors were in requiring 3 
process of care—‘‘failure to employ accepted stan- reviews 0 0 3 5 1 9(11%) 
dards of management.’ One error was in the ‘‘per- Number of cases 
formance of a procedure or operation,’’ and three with possible 
eae ; ; negligence 0 0 0 i 1 2 (3%) 
errors were ‘“‘primarily caused by the medical 
Table 6. Appendicitis case ratings on medical management causation and negligence 
Site 1 Site 2 Site 3 Site 4 Site 5 Totals 

Number of cases reviewed 7 9 44 51 12 123 
Number of cases with possible 

medical management causation 4 2 16 18 3 43 (35%) 
Number of cases requiring three 

reviews 1 1 6 4 0 12 (10%) 
Number of cases with possible 

negligence | 1 6 az 0 15 (12%) 


ip: 
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ing of the medical records and they occurred pri- 
marily at one site. Also, given the HMOs’ respon- 
sibility in paying for care, it seems unlikely that 
cases would not be in their files. 

If ‘‘true’’ cases were missed, it probably occurred 
because of problems in databases and data retrieval 
systems in the respective organizations. In fact, the 
number of UE cases identified at one site (Site 1 in 
Table 3) was suspiciously low. At the time of data 
collection, this site was employing a new data sys- 
tem which was proving to have some problems. A 
recent rerun of the data has yielded additional cases 
for the same period with a UE incidence rate much 
closer to that of the other five sites. Our general 
impression, based in part on evaluations of the 
additional cases from Site 1, is that any missing 


‘‘true cases’ are typical cases, and would have a. 


similar rate of causation and negligence as that 
found for the cases reviewed. 

The relatively low error rate found in this study 
suggests efficient targeting of UE cases based on the 
criteria. It is clear that criteria for some of the UEs, 
such as rectal cancer, which had only an 8% error 
rate, are tighter, more narrowly defined, and yield a 
more homogeneous group of cases than others. The 
strength of these more narrowly defined criteria 
have to be weighed against the usefulness of the 
data from the broader, and perhaps looser criteria 
developed. Despite the higher error rate and lower 
yield for negligence, cases such as those identified 
by the birth injury criteria are associated with 
enough risk to support use of broader criteria. 

Interestingly, as the authors were exploring the 
use of ICD-9 diagnosis and CPT procedure codes to 
target cases for their study, a similar approach was 
being explored by three HMOs and the Department 
of Health in Minnesota (Solberg et al. 1990). This 
group also had recognized the need to develop 
approaches that produced ambulatory cases with a 
high probability of reflecting inadequate prehospital 
care. They identified 15 ‘“‘indicator conditions’’ 
which met these criteria: 1) were clearly defined 
using ICD-9 codes, 2) had an admission rate of at 
least one per 10,000 enrollee years, 3) covered a 
broad range of patient demographic characteristics 
and care types, and 4) had a perceived likelihood 
that the indicator admission either should have oc- 
curred earlier or could have been avoided. The 
report of the Minnesota project does not indicate 
use of any additional qualifiers, such as length of 
stay or cancer staging, to further refine UE screens. 

Four of the UEs overlap or are identical with the 


Targeting Ambulatory Care Cases 


indicator conditions examined in the Minnesota 
project; both studies identified breast cancer, ap- 
pendicitis, ectopic pregnancy, and premature birth 
as potential highly productive areas for quality im- 
provement in ambulatory care. Several of the indi- 
cator conditions in the Minnesota project were con- 
ditions, such as diabetes mellitus and hypertension, 
where patient compliance played a critical role in 
clinical outcomes. This type of condition had been 
excluded from the current study in order to focus on 
UEs where the outcome was most likely to reflect a 
problem that could be addressed directly within the 
ambulatory care organization. This is not to say that 
UEs related to patient compliance are not important 
for quality assurance, or that the ambulatory care 
organization does not have responsibility to influ- 
ence compliance. 

The frequency with which the identified cases 
actually did not experience the indicator condi- 
tion—parallel to our validity—is not presented in 
the Minnesota report. However, in regard to utility, 
the Minnesota project found that 22% of the 673 
cases identified using the indicator conditions as 
screens were cases “‘representing probable quality 
of care concerns’’ (Solberg et al. 1990, p. 359). 
Quality of care concerns as described in the Min- 
nesota project were meant to be cases “‘that should 
be reviewed by a physician’’ (Solberg et al. 1990, 
p. 362) and probably were most similar to cases in 
our study where evidence of medical management 
causation was found. Their rate was slightly lower 
(22%) than that found in our study (31%). This may 
be accounted for by the differences between the 
UEs and indicator conditions, and the more exten- 
sive and complex criteria for the birth injury UE 
(including 12 ICD-9 diagnostic codes) in this study, 
compared to the Minnesota project’s single birth- 
related indicator condition of premature birth. The 
Minnesota project did not report on possible negli- 
gence in the cases with the indicator conditions. 

The successful application of the two indepen- 
dent efforts to develop screens using hospital codes 
to target ambulatory care cases for quality assur- 
ance and risk management supports the potential 
usefulness of this approach. The Minnesota project 
report did not comment on the potential usefulness 
of its approach for risk management in ambulatory 
care; in our study, identification of UEs was 
planned specifically with the interests of both risk 
management and quality assurance in mind. 

Estimates of the rates for management causation 
and negligence for typical births or all cases of 
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appendicitis are not available; therefore it is difficult 
to evaluate the yield or “‘productivity’’ of the UE 
screens. Donn et al. (1989) reviewed 115 birth cases 
being prepared for litigation and found that medical 
management causation contributed to negative birth 
outcomes in 45% of the cases, and that in 30% of the 
cases there was some question of negligence. These 
rates of causation and negligence are higher than 
those found in our study, but that is not surprising 
considering the source of Donn et al.’s (1989) study 
sample. 

Jess et al. (1981) examined the records of all 
patients who underwent an operation for acute ap- 
pendicitis over a six-month period at one site, and 
found that 16% of acute appendicitis cases had a 
perforated appendix, suggesting a delay in diagno- 
sis. The UE screen described in this study specifi- 
cally targeted appendicitis cases with peritonitis or 
abscess, and therefore the yield of 35% with man- 
agement causation, and 12% with deviation from 
standard of practice, is probably compatible with 
the findings of Jess et al. It is only possible to 
speculate regarding rates of causation and negli- 
gence for the other UEs included in our study, 
because the funding constraints precluded physi- 
cian reviews for all the UEs. 

Obviously, both an ambulatory care organiza- 
tion’s clinical population and its quality of care are 
determinants of the number of UE cases identified 
for a specific type of UE. As desired for the research 
study, use of the UE screens described earlier pro- 
duced a manageable number of cases. The specifi- 
cation of parameters for identification of a UE case 
(in terms of codes and days, etc.) could be relaxed 
or modified, so the number of incidents for review 
would increase. For example, rather than limiting 
postpartum complications to only those that led to 
an extension in length of hospitalization, one might 
want to include any complications which required 
procedures not normally part of routine postnatal 
care. The procedures included would be dictated 
by the particular goals and protocols of physicians 
and risk managers within the ambulatory care or- 
ganization. 

In addition, there are ambulatory care problems 
that occur relatively infrequently, but which war- 
rant identification and review whenever they occur. 
Two medical diagnoses for which hospitalization 
would be expected, and which are vulnerable areas 
in primary/ambulatory care, are torsion of the tes- 
ticle and bacterial meningitis. These conditions, in 
their early stages, can be treated very effectively, 
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but require a high index of suspicion on the part of 
the providers to assure early detection. Therefore, 
ambulatory care organizations might want to in- 
clude UE screens for these conditions, assuring 
recognition of problems needing immediate action 
to ameliorate any negative clinical and legal/finan- 
cial consequences, assuring feedback to keep prac- 
titioners alert, and assuring concurrent review of 
the quality of care provided. Although, for the 
purposes of the research, cases were identified ret- 
rospectively for one to five years, cases could be 
identified concurrently (on discharge or even, as 
one site suggested, during the admission authoriza- 
tion process). Use of the screens likely would be a 
dynamic process, with different types of cases tar- 
geted depending on the particular protocols or 
needs identified. 

In this research, physicians were employed for 
the reviews to establish the utility of the UE 
screens, the yield in terms of management causation 
and negligence. The UE screens are a method for 
limiting the number of cases requiring physician 
review for quality improvement, and therefore in- 
crease the feasibility of employing physicians for 
this purpose. However, physician review would not 
be essential for all purposes. For certain risk man- 
agement purposes, reviews of the identified cases 
by nurses or risk management staff using explicit 
criteria may be adequate. Further, for certain qual- 
ity assurance activities, an initial review by staff 
may be useful in further limiting the cases for phy- 
sician review or discussion. 

There has been a focus in this paper on yield, on 
the number of cases with medical causality and 
deviation from the standard of care. However, this 
should not be taken to suggest that the primary 
objective is searching for providers who are “‘bad 
apples.’’ Identification of problematic cases is use- 
ful in risk management and in quality improvement 
activities such as standard setting and continuing 
education. Even where case reviews yield differ- 
ences between reviewers, the information is useful 
for collaborative quality improvement processes 
because discussion of such cases can be valuable in 
both defining standards and reducing variation. 
Such discussions would reflect an interest in im- 
provement, not solely in placing blame. 


Summary 


Identification of highly productive areas, in terms of 
patient outcomes and an organization’s risk and 
quality of care concerns, which are manifested 
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through a hospitalization with a selected combina- 
tion of diagnoses and procedures, seems useful as a 
low-cost, focused methodology for ambulatory care 
organizations. This is not to suggest that the ap- 
proach described here is sufficient, that use of UE 
screens is all that an organization should undertake 
for risk management and quality assurance. How- 


Notes 


Support for this paper was provided by funds from the 
Agency for Health Care Policy and Research, ROI 
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1 The screening criteria for the other UEs are available 
from the authors. 

2 Given the small number of cases per UE type at each 
site, the cases have been combined across sites and/or 
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